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ABSTRACT 

Performance  metrics  for  human-robot  interaction  in  urban  search 
and  rescue  (USAR)  are  just  beginning  to  appear  in  the  literature  as 
researchers  try  to  establish  a  way  of  describing  and  evaluating 
human-robot  task  performance  in  this  high-risk,  time-critical 
domain.  In  this  paper  we  propose  that  human-robot  interaction 
metrics  should  focus  on  the  work  system  as  a  whole,  examining 
the  robot’s  effects  on  human  task  performance  within  the 
overarching  context  of  human  work.  Moreover,  these  effects 
should  be  examined  within  the  context  of  real-time  human 
performance  in  field  settings,  rather  than  in  simulation  or 
experimental  environments.  This  position  stems  from  a  basic 
assumption  that  we  are  interested  in  measuring  human-robot 
interaction  in  USAR  because  we  want  to  see  how  it  affects  and 
aids  human  performance  in  this  time  and  safety-critical 
environment.  We  present  a  methodology  for  collecting  data  in  the 
field  and  subsequent  analysis  using  the  Robot-Assisted  Search  and 
Rescue  Coding  System  (RASAR-CS),  specifically  developed  for 
this  domain.  The  RASAR-CS  allows  us  to  capture  1  jbasic  verbal 
and  non  verbal  communications  describing  the  task  and  how  it  is 
accomplished  (what  is  being  said,  by  who  to  whom);  2)situation 
awareness  information  requirements  (from  the  robot  and  other 
sources)  -  for  developing  and  maintaining  situation  awareness, 
including  the  ability  to  capture  changing  requirements  over  time; 
3)team  processes  enabling  coordinated  activities,  efficient 
communication  and  strategy  planning;  and  4)  human-robot 
interaction  in  tenns  of:  robot-operator  initiated  robot  activities,  and 
physical  interaction  with  robot. 

KEYWORDS:  human-robot  interaction,  performance 
metrics,  field  methodologies 


1.  INTRODUCTION 

Human-robot  interaction  in  the  Urban  Search  and  Rescue 
(USAR)  domain  is  a  field  of  study  that  has  drawn 
increasing  interest  in  light  of  the  use  of  robots  at  the  World 
Trade  Center  [5]  and  its  designation  as  a  benchmark  domain 
in  the  seminal  DARPA-NSF  study  on  Human-Robot 
Interaction  conducted  in  2001  [2].  Performance  metrics  for 
human-robot  interaction  in  USAR  are  just  beginning  to 
appear  in  the  literature  as  researchers  try  to  establish  a  way 
of  describing  and  evaluating  human-robot  task  performance 
in  this  high-risk,  time-critical  domain.  In  the  afore¬ 
mentioned  DARPA-N  SF  study,  simple  base  measures  were 


proposed:  the  ratio  of  persons  to  robots  (h-r  ratio),  spatial 
relationships  (commander,  peer,  teleoperator,  developer) 
and  authority  relationships  (supervisor,  operator,  peer  and 
bystander).  Some  of  the  metrics  proposed  subsequently 
focus  on  aspects  of  the  robot  system  exclusively,  (e.g.,  the 
interface)  or  on  aspects  of  human  performance  solely  in 
relation  to  working  with  the  robot  [8,  9,  14].  In  this  paper 
we  take  a  more  human-centric  position:  human-robot 
interaction  metrics  should  focus  on  the  work  system  as  a 
whole,  examining  the  robot’s  effects  on  human  task 
performance  within  the  overarching  context  of  human  work. 
This  position  stems  from  a  basic  assumption  that  we  are 
interested  in  measuring  human-robot  interaction  in  USAR 
because  we  want  to  see  how  it  affects  and  aids  human 
performance  (ultimately,  that  is  the  goal  for  measuring 
human-robot  interaction  in  any  work-related  field  or 
application). 

1 . 1  Field  Studies  in  USAR 

Field  studies  conducted  with  rescue  workers  offer  the  most 
valid  setting  in  which  to  study  human-robot  interaction. 
USAR  is  an  established  work  environment  offering 
opportunities  to  study  the  effects  of  introducing  robotic 
technology  into  a  workplace  and  occupation  with  existing 
goals,  tasks  and  processes.  It  is  arguably  one  of  the  first 
workplace  applications  where  robots  work  in  the  same 
spaces  with  people  whose  jobs  do  not  normally  involve 
robotics  to  perform  a  task  (Industrial  robots  are  usually 
separated  from  humans,  and  are  not  mobile).  Moreover, 
robots  have  been  used  in  real  disaster  responses,  and  are 
gradually  becoming  incorporated  into  USAR  training  both 
nationally  and  internationally.  Real-time  high  fidelity 
training  exercises  are  conducted  regularly  in  order  for 
USAR  task  force  members  to  attain  or  maintain  certification; 
these  exercises  offer  a  double  advantage  for  studying  HRI  in 
that  the  targeted  end-users  may  be  observed  performing  in 
realistic  work  environments.  USAR  task  forces  can  be 
characterized  as  extreme  teams  [11]  who  function  in 
dynamic,  high  risk,  time  critical  environments.  Team 
members  must  function  in  conditions  which  are  often 
physically,  mentally  and  emotionally  taxing.  Field  studies 
with  participants  who  are  truly  representative  of  this  user 
group  for  whom  the  technology  is  being  optimized  offer  the 
most  power  in  terms  of  generalizability. 
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1 .2  Focusing  on  Human  Performance 

Measures  of  human-robot  interaction  in  USAR  must  focus 
on  human  performance.  The  current  state  of  the  practice  in 
robot-assisted  search  and  rescue  is  teleoperation.  Though 
autonomous  and  semi-autonomous  robots  may  soon  be 
entering  the  workplace,  they  will  still  be  machines  designed 
to  perform  tasks  as  determined  by  a  person.  Robots  are  not 
conscious,  they  have  no  projects  of  their  own  other  than 
those  assigned  to  them.  Clancey  [7]  points  this  out  to 
illustrate  that  it’s  too  soon  to  talk  about  human-robot 
cooperation  or  collaboration:  instead,  robots  serve  as 
assistants  to  people  working  toward  a  project  goal. 

Therefore  the  measure  of  a  robot’s  usefulness,  efficiency 
and  functionality  is  based  solely  on  whether  it  contributes  to 
helping  a  person  (or  team)  accomplish  a  goal  by  making 
that  person’s  or  team’s  task  performance  more  efficient, 
effective,  or  easy  in  some  way.  This  means  measuring 
human  performance  (aided  by  robots)  is  the  key.  This  is 
different  from  the  position  taken  in  Drury  et  al.  [9]  that 
usability  requirements,  which  focus  primarily  on  the  robotic 
system,  are  the  most  appropriate  way  to  measure  human- 
robot  interaction.  We  believe  human-robot  systems  must  be 
examined  and  measured  in  terms  of  their  effect  on  human 
performance,  since  that  is  what  they  are  designed  to 
augment  or  improve. 

What  are  the  criteria  for  measures  of  robot-assisted 
human  performance  in  USAR?  In  this  domain  there  are 
established  goals:  search,  rescue  (extrication),  structural 
evaluation,  medical  assessment  &  treatment,  information 
transfer,  command  &  control,  and  logistics.  Blitch  [1] 
pointed  out  the  potential  applications  of  robots  in  tunnel  and 
confined  space  search:  now,  it  is  evident  that  there  are  many 
more  tasks  in  which  robots  may  play  a  part  in  USAR,  e.g. 
medical  reachback,  shoring,  communications  &  information 
transfer,  and  safety  monitoring.  Past  experience  shows  that 
new  technologies  evolve  when  they  reach  the  workplace, 
and  many  times  end  up  performing  tasks  or  serving 
purposes  for  which  they  were  not  originally  intended.  What 
we  can  do  is  identify  tasks  as  they  emerge,  study  the 
human-robot  interaction  processes  and  determine  optimal 
task  allocation  and  roles,  understanding  that  this  is  an 
iterative  process  that  will  change  as  the  technology 
advances.  Based  on  these  tasks,  we  can  measure  human- 
robot  performance  both  individually  (one  person  operating  a 
robot)  and  in  teams  (more  than  one  person  operating  a  robot 
or  robots). 

Our  field  research  has  shown  that  situation  awareness 
and  team  processes  are  two  constructs  which  relate  to 
human  performance  when  working  with  robots  [3,  5,  6], 
Situation  awareness  (SA)  as  defined  by  Endsley  [10]  is 
“. .  .the  perception  of  the  elements  in  the  environment  within 
a  volume  of  time  and  space,  the  comprehension  of  their 
meaning  and  the  projection  of  their  status  in  the  near  future” 
(p.  97).  Our  studies  have  shown  situation  awareness  to  be 
related  to  performance  and  that  most  of  the  operators’  time 


is  spent  gathering/maintaining  situation  awareness  [3,4]. 
Operators  with  high  situation  awareness  ratings  were  better 
performers  in  our  study  of  28  robot  operators  [4].  Team 
processes  are  also  related  to  operator  performance; 
operators  who  talked  more  with  their  teammates  about  goal- 
directed  aspects  of  the  task  had  higher  situation  awareness 
ratings  and  found  the  victim  more  often.  There  is  an 
interactive  affect  between  situation  awareness  and  team 
process — suggesting  operators  who  talk  more  with  their 
tether  manager  or  teammate  are  better  at  building  a  mental 
model  of  the  robot  as  it  functions  in  the  void  space,  and  also 
are  better  at  building  a  shared  mental  model  of  the  search. 
Research  on  teams  and  mental  models  has  suggested  that 
having  a  shared  mental  model  of  the  problem  space  can 
increase  situation  awareness  and  team  performance  [11,  16]. 
Effective  planning  and  communication  strategies  were 
found  to  increase  team  shared  mental  models  and 
correspondingly  team  performance.  Therefore,  human-robot 
interaction  in  USAR  needs  to  be  measured  not  only  at  the 
individual  level,  but  also  at  the  team  level. 


2.  METHODOLOGY 

In  this  section  we  present  our  methods  of  field  data 
collection  and  data  analysis,  including  a  description  of  the 
Robot-Assisted  Search  and  Rescue  Coding  System 
(RASAR-CS). 

2.1  Data  Collection  in  the  Field 

Data  collection  is  an  observational  procedure,  where  the 
researcher  is  present  during  the  user-robot  interaction, 
though  not  an  active  participant.  We  tape  the  interaction 
using  2-4  cameras  depending  on  the  environment  (Table  1). 
Minimally,  one  camera  records  the  robot’s  eye  view  directly 
from  the  operator  control  unit  (OCU),  and  a  second  camera 
records  the  operator  (making  sure  to  have  a  clear  view  of 
the  face)  as  she  works  with  the  robot.  When  environmental 
conditions  permit,  we  set  up  a  third  camera  on  a  tripod  to 
record  the  operator’s  hands  manipulating  the  OCU,  and  a 
fourth  camera  to  record  an  external  view  of  the  robot  when 
available.  Depending  on  the  environment  and  the  number  of 
personnel  available  for  the  data  collection  process,  some 
cameras  may  be  fixed  on  tripods;  however,  in  USAR 
conditions,  most  of  the  time  views  2-4  must  be  handheld 
due  to  lack  of  level  spots  for  setting  up  a  fixed  camera. 
Video  recordings  of  the  operators  manipulating  the  robot, 
the  robot’s  eye  view,  and  the  available  external  views  are 
edited  and  synchronized  to  create  tapes  with  2  views  side- 
by-side.  These  videotapes  are  then  used  to  code  statements 
and  gestures  made  by  both  the  operators  and  surrounding 
personnel,  and  robot  movements.  Trained  raters  code  the 
videotapes  using  the  Noldus  Observer  Video-Pro  [13] 
observational  coding  software. 


Camera 

No. 

View 

Setup 

1 

Robot’s  eye  view 

Attached  to  OCU 

2 

Operator  view 

Tripod  or  handheld 

3 

Operator-OCU  view 

Tripod  or  handheld 

4 

Robot-external  view 

Tripod  or  handheld 

Table  1.  Camera  views  for  human-robot  interaction 
field  research  in  USAR. 


form,  content  and  mode  of  communication.  Our  goal, 
however,  is  two-fold,  not  only  to  capture  the  “how”  and 
“what”  of  USAR  robot  operator  teams,  but  also  the  “who”, 
and  to  capture  observable  indicators  of  robot  operator 
situation  awareness.  In  addition,  in  order  to  adhere  to  the 
tenets  of  IA,  the  framework  must  be  extended  to  include 
examination  of  physical  interactions  with  the  robot  system(s) 
in  the  environment. 


2.2  Video-based  Interaction  Analysis 

What,  then,  are  some  appropriate  measures  and  metrics  for 
USAR  human-robot  interaction?  Primary  human 
performance  outcome  measures  for  search  at  the  most  basic 
level  include:  Was  a  victim  found,  how  long  did  it  take, 
were  any  victims  missed,  were  important  cues  noticed  (heat, 
color,  objects,  information  synthesis  with  knowledge  about 
the  event  &  environment)?  Other  measures  that  are  related 
to  these  primary  outcomes  specifically  measure  situation 
awareness  and  team  processes.  These  measures  are  gathered 
via  a  video-based  Interaction  Analysis  technique  for 
investigating  HRI  in  rescue  robotics.  Interaction  Analysis 
(IA)  is  an  interdisciplinary  approach  to  studying  the 
interaction  of  humans  with  each  other  and  with  objects  in 
their  environment.  Jordan  &  Henderson  [12]  assert, 
“Video-based  Interaction  Analysis  is  a  powerful  tool  in  the 
investigation  of  human  activity  that  is  particularly  effective 
in  complex,  multi-actor,  technology-mediated  work 
settings  ...  It  is  currently  undergoing  a  period  of  rapid 
development,  driven,  in  part,  by  researchers'  dissatisfaction 
with  conventional  methods,  and  in  part  by  the  ubiquity  of 
video  equipment.”  (p.44) 

The  goal  of  Interaction  Analysis  is  to  identify 
regularities  in  the  ways  in  which  participants  utilize  the 
resources  of  the  complex  social  and  material  world  of  actors 
and  objects  within  which  they  operate.  To  do  this  we  must 
examine  two  components  of  IA,  which  are  intertwined,  but 
distinct  as  well:  human-human  interaction,  and  human- 
object  interaction.  Interaction  Analysis  assumes  that 
knowledge  and  action  are  fundamentally  social  in  origin, 
organization,  and  use.  Knowledge  is  seen  as  located  in  the 
interactions  between  people  engaged  with  the  material 
objects  in  their  surroundings;  therefore  communication 
analysis  plays  an  important  role  in  Interaction  Analysis  as  a 
means  of  analyzing  human-human  interactions. 

Although  variety  of  approaches  to  examining 
communication,  we  chose  the  FAA’s  Controller-to- 
Controller  Communication  and  Coordination  Taxonomy 
(C4T)  [15]  framework  as  the  starting  point  for  the 
development  of  our  communication  analysis  system 
designed  to  assess  HRI  in  rescue  robotics.  The  C4T  uses 
verbal  information  to  assess  team  member  interaction  from 
communication  exchanges  in  an  air  traffic  control 
environment.  We  used  the  C4T  model  because  it  captures 
the  “how”  and  “what”  of  team  communication  by  coding 


2.3  Robot-Assisted  Search  and  Rescue  Coding 
System  (RASAR-CS) 

A  methodology  to  capture  and  assess  robot  assisted  task 
performance  in  rescue  robotics  must  consider  both  human 
team  member  interactions  (robot  operator  and  other  team 
members),  and  human  -  robot  interactions.  To  meet  the 
goals  of  a  methodology  capable  of  defining  robot  assisted 
tasks,  and  examining  SA  and  teamwork  defined  earlier,  we 
developed  the  Robot-Assisted  Search  and  Rescue  Coding 
System  (RASAR-CS).  The  RASAR-CS  captures  Basic 
verbal  and  non  verbal  communications  describing  the  task 
and  how  it  is  accomplished  (what  is  being  said,  by  who  to 
whom);  Situation  Awareness  Information  requirements 
(from  the  robot  and  other  sources)  -  for  developing  and 
maintaining  situation  awareness,  including  the  ability  to 
capture  changing  requirements  over  time;  Team  processes 
enabling  coordinated  activities,  efficient  communication 
and  strategy  planning;  and  Human-Robot  interaction  in 
terms  of:  Robot-operator  initiated  robot  activities,  and 
Physical  interaction  with  robot. 

Following  the  Interaction  Analysis  approach,  the 
RASAR-CS  consists  of  four  main  coding  components 
enabling  analysis  of  SA  and  team  factors  through  human- 
human  interaction  and  human  robot  interaction.  These 
components  include  verbal  communication,  communication 
medium,  nonverbal  interaction  and  robot  movements. 

2.3.1  Human-Human  Verbal  Communication 

The  verbal  communication  analysis  codes  team  member 
statements  across  four  categories:  1)  Speaker-recipient  dyad 
-  who  is  speaking  to  whom,  2)  Content  or  topic  of  the 
communication  3)  Statement  form  or  grammatical  structure 
of  the  communication,  4)  Function  or  intent  of  the 
communication  (Table  2).  By  examining  dyad,  content,  and 
form,  we  can  examine  task  procedures  and  team 
coordination.  Similarly,  content  and  function  provide 
indicators  of  operator  situation  awareness. 

Speaker-recipient  dyad.  Based  on  review  of  the  search 
task  videotapes,  potential  conversants  included  the  operator, 
tether  manager,  team  member,  the  group,  and  the  robot 
specialist/researcher.  Dyad  codes  indicate  the  speaker, 
followed  by  the  recipient.  For  example,  “operator-tether 
manager”  indicates  a  statement  was  made  by  the  operator 


Category 

Subcategories 

Definitions 

Human-Human  Verbal  Communication 

Sender/Recipient 

Dyad 

Operator-tether  manager 

Operator:  individual  teleoperating  the  robot 

Tether  manager-operator 

Tether  manager:  individual  manipulating  the  tether  and  assisting  operator  with  robot 

Team  member-operator 

Team  member:  one  other  than  the  tether  manager  who  is  assisting  the  operator  (usually  interpreting) 

Operator-  team  member 

Researcher-operator 

Researcher:  individual  acting  as  scientist  or  robot  specialist 

Operator-re  searcher 

Other-operator 

Other:  individual  (not  tether  manager,  team  member,  or  researcher)  interacting  with  the  operator 

Operator-other 

Operator-group 

Group  -set  of  individuals  interacting  with  the  operator 

Content 

Robot  state 

Robot  functions,  parts,  errors,  capabilities,  etc. 

Environment 

Characteristics,  conditions  or  events  in  the  search  environment 

Information  synthesis 

Connections  between  current  observation  and  prior  observations  or  knowledge 

Robot  situatedness 

Robot’s  location  and  spatial  orientation  in  the  environment;  position 

Victim 

Pertaining  to  a  victim  or  possible  victim 

Navigation 

Direction  of  movement  or  route 

Search  strategy 

Search  task  plans,  procedures  or  decisions 

Off  task 

Unrelated  or  extraneous  subject 

Statement  Form 

Question 

Request  for  information 

Instruction 

Direction  for  task  performance 

Comment 

General  statement,  initiated  or  responsive,  that  is  not  a  question,  instruction  or  answer 

Answer 

Response  to  a  question  or  an  instruction 

Function 

Non-operator 

Default  for  statements  made  by  individuals  other  than  the  operator 

Seek  information 

Asking  for  information  from  someone 

Report 

Sharing  observations  about  the  robot,  environment,  or  victim 

Clarify 

Making  a  previous  statement  or  observation  more  precise 

Confirm 

Affirming  a  previous  statement  or  observation 

Convey  uncertainty 

Expressing  doubt,  disorientation,  or  loss  of  confidence  in  a  state  or  observation 

Plan 

Projecting  future  goals  or  steps  to  goals 

Provide  information 

Sharing  information  (other  than  report)  in  response  to  a  question  or  offering  unsolicited  information 

Team 

Communication 

Coordination 

Team  members  coordinate  actions  to  synchronize  specific  proximal  task  activities 

Planning 

Planned  strategies  for  future  goal  accomplishment 

Source  of 
Information  used 
in  discussion 

Audio 

Verbal  information  or  information  from  previous  dialog 

Visual  image 

Robot  image  or  information  from  image  provides  the  basis  for  statement 

Sensor 

Sensor  or  information  from  sensor  provides  the  basis  for  statement 

Human-Robot  Interaction  (Nonverbal  interaction  via  the  robot) 

Physical 

orientation 

Ear  to  robot 

Ear  is  directed  toward  the  robot 

Eye  to  robot 

Turning  so  that  the  human  looks  at  the  robot 

No  verbal  communication 

No  verbal  communication  with  the  operator 

Gestures 

Come  forward 

Motioning  toward  the  robot  to  move  forward 

Thumbs  up 

Closing  the  fist  with  the  thumb  extended  upward 

Stop 

Holding  up  a  hand  with  the  palm  toward  the  r 

Pointing 

Using  fingers  to  point  in  a  particular  direction  or  at  a  specific  object 

“OK”  sign 

Closing  the  thumb  and  forefinger  in  a  circle  indicating  the  “OK”  sign 

Other 

Other  gestures  (usually  conversational  with  no  intended  message) 

Interaction  with 
Robot 

Clean  lens 

Cleaning  the  robot  camera  lens 

Move/shift 

Altering  the  position  of  the  robot 

Pick  up 

Lifting  the  robot  off  the  surface  upon  which  it  is  moving 

Other 

Other  physical  contact  with  the  robot 

Robot  Movement 

Moving 

Forward  or  backward  locomotion 

Stationary 

No  movement  at  all 

Panning 

Rotating  side  to  side  without  forward  movement,  or  manipulating  the  camera  lens  up/down 

Table  2.  RASAR-CS  (for  USAR  search  task) 


and  directed  toward  the  tether  manager  (Note:  the  code 
“tether  manager  -  operator”  indicates  the  tether  manager 
initiated  the  communication  with  the  operator). 

Content.  Seven  elements  representing  the  content  were 
generated:  1-  Statements  related  to  robot  functions,  parts, 
errors,  or  capabilities  (Robot  state),  2-  Statements 
describing  characteristics,  conditions  or  events  in  the  search 
environment  (Environment),  3-  Statements  reflecting 
associations  between  current  observations  and  prior 
observations  or  knowledge  (Information  synthesis),  4- 
Statements  surrounding  the  robot’s  location,  spatial 
orientation  in  the  environment,  or  position  (Robot 
situatedness),  5-  Indicators  of  direction  of  movement  or 
route,  (Navigation),  6-  Statements  reflecting  search  task 
plans,  procedures  or  decisions  (Search  Strategy),  and  finally 
7-  Statements  unrelated  to  the  task  (Off  Task).  The  first 
four  content  elements  are  relevant  to  building  and 
maintaining  SA  in  search  operations,  while  the  elements  of 
navigation  and  search  strategy  require  SA. 

Form.  Similar  to  the  C4T  taxonomy,  the  form  category 
contains  the  elements:  1-  Question  (request  for  information), 
2-  Instruction  (direction  for  task  performance),  3-  Comment 
(general  statement,  initiated  or  responsive,  that  is  not  a 
question,  instruction  or  answer)  and  4-  Answer  (response  to 
a  question  or  an  instruction). 

Function.  Function  refers  to  the  intent  of  the 
communication  -  elements  include:  1-  Seek  information 
(asking  for  information  from  someone),  2-  Report  (sharing 
observations  about  the  robot  or  environment),  3-  Clarify 
(making  a  previous  statement  or  observation  more  precise), 

4-  Confirm  (affirming  a  previous  statement  or  observation) 

5-  Convey  uncertainty  (expressing  doubt,  disorientation,  or 
loss  of  confidence  in  a  state  or  observation),  6-  Plan 
(projecting  future  goals  or  steps  to  goals),  7-  Provide 
information  (sharing  information  other  than  that  described 
in  report,  either  in  response  to  a  question,  or  offering 
unsolicited  information). 

The  function  elements  of  reporting  and  providing 
information  merit  explanation,  as  they  appear  very  similar. 
Reporting  involves  perception  and  comprehension  of  the 
robot’s  state  or  situatedness,  the  environment  or  information 
synthesis.  Any  other  information  shared  by  an  operator,  in 
answer  to  a  question  or  on  his  own,  is  classified  as 
providing  information  (for  example  search  strategy  or 
navigation).  Indicators  of  SA  are  captured  in  the  function 
category  primarily  through  the  elements  reporting  and 
planning.  When  operator  shares  information  (reports)  based 
on  the  robot’s  eye  view,  we  can  infer  the  first  two  levels  of 
SA,  perception  and  comprehension,  have  taken  place.  The 
third  SA  level,  planning  and  projection,  is  captured  in  the 
function  category  as  the  element  “plan.” 

2.3.2  Team 

Communication.  Team  communication  offers  insights  into 
how  goals  are  accomplished.  Categories  include: 


1-  Coordinating  activities  (to  synchronize  specific  proximal 
task  activities)  and  2-  Planning  (for  future  goal 
accomplishment). 

Medium.  Team  communication  is  also  coded  according 
the  medium  used  to  in  conveying  information:  1-  Visual 
(visual  image  provided  the  foundation  for  the 
communication),  2-  Auditory  (verbal  information  provided 
the  foundation  for  the  communication),  and  3-  Sensor 
(sensor  provided  the  foundation  for  the  communication). 

2.3.3  Human  -  Robot  Interaction 

Nonverbal  interaction  with  robot.  Nonverbal  HRI  includes 
nonverbal  communication  between  humans  via  the  robot 
camera,  and  physical  interaction  of  humans  with  the  robot. 
When  robots  are  co-located  with  humans,  humans 
physically  orient  to  the  robot  and  use  gestures  when 
communicating  with  the  operator  in  control  of  the  robot. 
Additionally,  humans  have  the  ability  to  physically  touch  or 
interact  with  the  robot  to  cooperatively  accomplish  goals. 
The  three  main  nonverbal  categories  include:  physical 
orientation,  gestures,  and  physical  interaction  with  co¬ 
located  robot. 

Physical  orientation.  Physical  orientation  includes 
positioning  the  body  during  communication  with  the  robot 
operator  so  that  the  1-  Ear  is  directed  toward  the  robot  (ear 
to  robot),  and  2-  Turning  so  that  the  human  looks  at  the 
robot  (eye  to  robot). 

Gestures.  Again,  while  communicating  with  the  robot 
operator,  gesture  can  be  used  to  convey  meaning  to  the 
operator  via  the  robot  camera.  Gestures  include:  1-  Come 
forward  (motioning  toward  the  robot  to  move  forward),  2- 
Pointing  (using  fingers  to  point  in  a  particular  direction  or  at 
a  specific  object),  3-  Thumbs  up  (closing  the  fist  with  the 
thumb  extended  upward),  4-  Stop  (holding  up  a  hand  with 
the  palm  toward  the  robot),  and  5-  OK  (closing  the  thumb 
and  forefinger  in  a  circle  indicating  the  “OK”  sign). 

Physical  Interaction  with  Robot.  Physical  interaction 
codes  include:  1-  Clean  lens  (cleaning  the  robot  camera 
lens),  2-  Move/shift  (altering  the  position  of  the  robot),  and 
3-  Pick  up  (lifting  the  robot  off  the  surface  upon  which  it  is 
moving). 

Robot  Movement.  The  three  major  robot  movement 
coding  categories  of  the  RASAR-CS  include:  1-  Moving 
(traveling  forward  or  back),  2-  Stationary  (no  movement  at 
all)  and  3-  Panning  (turning  from  side  to  side  without 
forward  or  backward  movement). 


3.  CONCLUSIONS 

We  have  presented  a  field  methodology  for  examining 
human-robot  interaction  in  USAR  which  focuses  on  robot- 
assisted  human  performance.  Using  a  video-based 
Interaction  Analysis  technique,  we  examine  both  human- 


human  interaction  and  human-robot  interaction  with 
measures  designed  to  capture  performance  of  human-robot 
systems.  The  Robot-Assisted  Search  and  Rescue  Coding 
Scheme  enables  us  to 

•  Examine  archival  videotaped  data.  Video  data 
involving  users  provides  a  richness  of  information 
that  we  previously  had  no  established  means  of 
harvesting. 

•  Decompose  novel  robot  assisted  tasks. 
Understanding  how  USAR  personnel  use  robots  to 
accomplish  tasks  provides  the  foundation  for 
developing  a  model  of  robot  assisted  task 
performance,  which  can  be  used  for  defining  best 
practices  and  generating  field  training. 

•  Identify  task  specific  SA  requirements  and  effective 
modalities  for  information  transfer  among  team 
members  for  use  in  system  design  (e.g.,  operator 
control  unit  interfaces,  and  web  pages  for  remote 
team  members). 

•  Evaluate  requirements  for  team  performance  such 
as  shared  mental  models,  coordination  of  activities, 
and  patterns  of  cooperative  behavior. 

•  Obtain  quantifiable  SA  and  team  data  for 
evaluating  effective  performance. 

•  Adapt  and  respond  to  changing  task  and 
technology’  requirements.  The  RASAR-CS  can  be 
reconfigured  to  meet  needs  of  various  tasks  and  to 
be  responsive  to  changes  in  technology  as 
advances  in  robotics  occur. 

The  RASAR-CS  allows  researchers  to  decompose 
both  human-robot  and  human-human  interaction  in  a 
meaningful  way  to  define  robot  assisted  task  performance 
including  task  procedures,  situation  awareness  requirements, 
and  team  process  and  coordination.  The  system  can  be 
applied  across  tasks  and  domains  by  utilizing  the  procedures 
outlined  for  modifying  the  relevant  codes.  In  assessing 
complex  environments  it  is  important  to  use  multiple 
methods  of  assessment.  The  RASAR-CS  is  an  effective 
methodology  to  add  to  researchers’  HRI  toolkit  for  analysis 
of  archival  videotapes  of  field  data,  or  used  as  a 
complement  to  other  techniques,  e.g.  onsite  expert  ratings  of 
situation  awareness  and  team  process,  self  ratings  of 
situation  awareness  and  team  process,  and  user  ratings  of 
traditional  evaluative  components  (usefulness,  ease  of  use, 
effectiveness,  satisfaction)  for  using  the  robot. 
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